Modern research workflows

12.06.2019

Introduction

Results of our little survey:

Name	Field	Research
Henrik	Bibliometrics	Statistics, coding, prose

Topics du jour:

The modern research cycle
The Open Science movement
Incorporating workflow thinking into your research

Part I: The modern research cycle

An idealised research project

As a researcher, in many ways, this is how you would want the ideal research project to look:

Requirements

…but then, you are not the one with the money (yet).

Many research funders now require the development of a heap of auxiliary information about the project:

Data management plan
Publication plan
Dissemination plan

Data management plan

Research data should be shared and reused more widely […] Better access to research data can boost innovation and value creation by enabling actors outside the research community to find new areas of application.

National strategy on access to and sharing of research data

What do you collect?
How do you treat it?
How will you keep/share it?

Publication plan

Where do you plan to publish?
What part of the project will make it into which publications?
How do the publications fit into the overall project?

The publishing cycle

The publishing cycle, really

Dissemination plan

How will you present your research?
In which channels?

Social media

A more realistic project plan

Is this you?

Charlie Chaplin (1916) The Floorwalker, Lone Star Corporation

Why all this stuff?

[W]e have two major points to consider. First, due to a lack of adequate incentives in the reward structure of professional science […] actual replication attempts are rarely carried out. Second, to the extent that they are carried out, it can be well-nigh impossible to say conclusively what they mean, whether they are “successful” (i.e., showing similar, or apparently similar, results to the original experiment) or “unsuccessful” (i.e., showing different, or apparently different, results to the original experiment).

Earp, B. and D. Trafimov (2015) Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology

This is the whole abstract of an interesting paper in the field of genomic biology:

The spreadsheet software Microsoft Excel, when used with default settings, is known to convert gene names to dates and floating-point numbers. A programmatic scan of leading genomics journals reveals that approximately one-fifth of papers with supplementary Excel gene lists contain erroneous gene name conversions.

Ziemann, M., Y. Eren, A. El-Osta (2016) Gene name errors are widespread in the scientific literature. Genome Biology 17:177

Storytime

In 2009, I did some data collection in connection with my doctoral research regarding people’s habits and considerations around the availability and price of electricity. The department had secured some external funding from various industry partners, and spent almost 250.000 NOK soliciting answers from 1500 Norwegians on a survey on this topic. To achieve 1500 answers they had to ask more than 22.000 people….

Once the data was in, I spent a lot of time analysing it and producing graphs that could be used in a publication. It was all based on the SPSS software for statistical analysis. SPSS very handily provides users with a menu-based graphical user interface that allows them to specify the parameters of statistical analysis without bothering them with the specifics of how the inputs are defined.

It also creates a log file that can be used to recreate the procedure of several of these analysis steps at some future point in time. The only problem is that this log file of the scrips run under the hood is not brought to the attention of the researcher. So I never spent too much time worrying about how the data should be interpreted by someone else at some point in the future. The result is that no effort was spent in making legible the variables and any operations that were performed on them in order to produce the results that I based my publication on.

Eight years later, a researcher looking to do a meta-analysis on a collection of such survey data contacted me, asking me for the underlying data and some explanation of the relevant variable definitions. So I went looking for the data, and finally found them in a subfolder on my (now mostly inactive) Dropbox account.

Here are some rows of some of the columns:

s4	s6	s7	s8	s9
4	4	1	NA	46
3	1	1	NA	125
3	1	1	NA	90
3	3	1	NA	156
4	5	1	NA	78

Only problem: I don’t know where I put the codebook!

An all-too familiar story

Hanson, K. A. Surkis and K. Yacobucci (2012) Data Sharing and Management Snafu in 3 Short Acts CC-BY

Part II: How to deal with this?

Just don’t do it

Research habits change very slowly
- Being too far ahead of the pack can be more work than it’s worth
Funder mandates are actually minimal

Try to integrate the issues into your workflow

Can I set up my workflow in a way that is

resource-efficient
modular

Part III: Examples of digital workflows

The general idea

Collaborating

Always updated, always available for everyone?
- Sharepoint (requires institutional credentials)
- Overleaf (becomes better with knowledge of the world’s worst markup language, LaTeX)
- Jupyter notebooks / RMarkdown (become better with coding knowledge)

Keeping track

Version control software
- Git

Documenting

If you treat your

Sharing

The trade-offs

There are powerful, efficient tools at our disposal
There is a learning curve of varying steepness
Maybe

Resources

Foster Open Science, the EU educational resource for Open Science.

Open Science MOOC, coursework on Open Science principles and tools.

NTNU policies on Open Access publishing and Open Data.

National policies on the same.